In computing systems, the advancement of a program from one stage to the next often depends on the direct interaction of a user with the system at different stages. For example, the system may automatically perform one or more transactions at a first stage. Once the transactions are completed, then a user may need to review the results and interact with the system to initiate the next stage, by providing additional input (e.g., pressing a key on the keyboard).
If the length of time associated with the computations involved in the series of transactions is undesirably long, then a user will have to either wait until the end of the transactions is reached or alternatively move on to another program. Fortunately, automation programs are available that can be used to ease the burden on the user by automatically detecting the end of a transaction and advancing the system to the next stage. The interactions are usually repeating tasks, which can be performed with more accuracy and efficiency via automation.
Usually, an automation framework is configured to generate a predetermined sequence of user interface events (e.g., keyboard/mouse input) once the end of a computing transaction is detected, so that the system can move on to the next stage. In addition the automation framework needs to harvest information from various stages. However, determining the end of a computing transaction can be difficult depending on the nature of the transaction, the underlying system design or the configuration of the computing program.
Particularly, detecting the end of a transaction in a graphical user environment may be very difficult when working through a visual interface that only allows sampling activity on the display screen, in contrast to environments where detecting the end of transaction may be accomplished by checking the value of a variable. For example, in an environment where HTML content is received from a HTTP server, detection of the end of a transaction may be simply determined by checking the value of a flag that provides the receipt status of the content requested. In such environments, monitoring the exact appearance of various content on the display screen is not necessary for detecting the end of a transaction.
Current approaches for detecting an end of transaction include monitoring screen refresh events and updates or waiting until a certain time period passes without any updates being detected. Presence of animation or blinking characters on the screen typically prevents the screen from settling, however. As such, if the end of a transaction is not followed by a quiet period when the screen is not refreshed, the end of transaction may not be easily detected by the automation framework.
Other methods have been proposed that involve screen capture, optical character recognition (OCR) of screen content, and relatively sophisticated hashing algorithms or synchronization hooks that need to be included in program code. These methods, however, typically require the advance knowledge of the exact or the entire content of the screen to be successful.